Generic Discrimination: Partitioning and Sorting of Complex Data in Linear Time

We introduce the notion of discrimination, which is a generalized form of partitioning, and present an expressive term language for defining equivalence relations on complex data. The language allows definition of equivalence relations by freely combining structural equivalence, equivalence of lists under commutativity, idempotence (bag and set equivalence), and more. We then show that worst-case linear-time discriminators can be defined generically, by induction on the term language, using multiset discrimination. By employing discriminators for base types such as characters and integer segments that sort their inputs, it can be shown that the inductive construction yields discriminators that both partition and sort their input in linear time for a wide range of total preorders. This amounts to generically bootstrapping pigeonhole sorting for a finite segment of primitive data to linear-time sorting of complex data. We show how these discriminators, both sorting and nonsorting, can be coded up compactly and elegantly using Generalized Algebraic Data Types (GADTs) and list comprehensions and give some examples of applications of the use of discriminators. Finally, we argue that discrimination should replace equality testing as a language primitive for built-in types and abstract types that wish to only make equality observable: they algorithmically generalize equality testing, which is basically just discrimination of 2 elements. Discriminators allow partitioning and even sorting of arbitrary size lists in linear time without comparison operations (as in comparison-based sorting) or arithmetization of the values (as for hash-based methods). Thus even references in ML could be discriminated in linear time instead of quadratic time, if discrimination were the built-in operation for exposing reference equality, not equality testing. 1 Partitioning and discrimination Definition 1 (Equivalence, Partitioner, Discriminator). An equivalence (T,Eq) is a type T together with a binary relation Eq ⊆ T × T that is reflexive, symmetric and transitive. A partitioner for equivalence (T,Eq) is a function PartEq : [T ]→ [[T ]] that takes a list of elements and partitions them according to equivalence Eq. We call each element of the output a block (which is a list itself). More precisely, we have: